Coding Human Lip Motions with a Learned 3D Model
نویسندگان
چکیده
The lips are a critical factor in spoken communication and expression. Accurately tracking and synthesizing their motions from arbitrary head poses is essential for high-quality video coding. Our approach is to build and train 3D models of lip motion to compensate for the limited information available during tracking. We use physical models as a prior and combine them with statistical models, showing how the two can be smoothly and naturally integrated into a synthesis method and a MAP estimation framework for tracking. Because the resulting description has a small number of parameters, it is ideal for coding as well. We show how our methods allow us to accurately recover the 3D lip shape from raw 2D video data and resynthesize this shape with a small number of parameters.
منابع مشابه
3D Modeling and Tracking of Human Lip Motions
We address the problem of tracking and reconstructing 3D human lip motions from a 2D view. This problem is challenging due both to the complex nature of lip motions and the minimal data available from a raw video stream of the face. We counter both of these di culties with statistical approaches. We rst build a physically-based 3D model of lips and train it to cover only the subspace of lip mot...
متن کاملVisual Coding and Tracking of Speech Related Facial Motions
We present a new method for video-based coding of facial motions inherent with speaking. We propose a set of four Facial Speech Parameters (FSP): jaw opening, lip rounding, lip closure, and lip raising, to represent the primary visual gestures in speech articulation. To generate a parametric model of facial actions, first a statistical model is developed by analyzing accurate 3D data of a refer...
متن کامل3D Hand Motion Evaluation Using HMM
Gesture and motion recognition are needed for a variety of applications. The use of human hand motions as a natural interface tool has motivated researchers to conduct research in the modeling, analysis and recognition of various hand movements. In particular, human-computer intelligent interaction has been a focus of research in vision-based gesture recognition. In this work, we introduce a 3-...
متن کاملAnalysis of 3D Passive Walking Including Turning Motions for the Finite-width Rimless Wheel
The focus of studies in the field of passive walking has often been on straight walking, while less attention has been paid to the field of turning motions. In this paper, the passive motions of a finite width rimless wheel as the simplest 3D model of passive biped walkers was investigated with a focus on turning motions. For this purpose, the hybrid model of the system consisting of continuous...
متن کاملFrom Passive Dynamic Walking to Passive Turning of Biped walker
Dynamically stable biped robots mimicking human locomotion have received significant attention over the last few decades. Formerly, the existence of stable periodic gaits for straight walking of passive biped walkers was well known and investigated as the notion of passive dynamic walking. This study is aimed to elaborate this notion in the case of three dimensional (3D) walking and extend it f...
متن کامل